Simultaneous readout of CMOS APS imagers

ABSTRACT

A new method of reading an imager is achieved. The method comprises providing an imager array comprising n rows and m columns where a pair of rows can be read during a single row access time. A first image field is completed by sequentially reading and storing pixel values of pairs of adjacent rows of the imager array. The reading begins at a first row, and the reading stops when less than three rows are unread. Thereafter pixel values of the next row are read and not stored. Thereafter pixel values of the first row of the imager array are read and not stored. A second image field is completed by sequentially reading and storing pixel values of pairs of adjacent rows. The reading begins at the second row, the reading stops when less than two rows are unread.

This application claims priority to U.S. Provisional Application serialNo. 60/450,090 filed on Feb. 26, 2003, and herein incorporated byreference.

BACKGROUND OF THE INVENTION

(1) Field of the Invention

The invention relates to imaging arrays, and, more particularly, to amethod to an imager having dual row access capability.

(2) Description of the Prior Art

Standard CMOS Active Pixel Sensors (APS) utilize a row scanningarchitecture for addressing pixel data and for generating an imageframe. By sequentially scanning each row of the imager, a full image isgenerated in what is known as a progressive scan format. Color imagesuse standard mosaic filter patterns that are placed directly over theimaging pixels to break up the image into its basic color components.Primary color filters break up the image into the red, green, and bluecomponents while complimentary filters break up the image into yellow,cyan, and magenta components.

Standard patterns are commonly used to enable standardized productionmethods to be used. The most common standardized pattern is the Bayerpattern 25 shown in FIG. 2. As shown in FIG. 2, the Bayer pattern is anarray comprising 2×2 pixel sub-arrays. Each 2×2 sub-array 25 comprises,from left to right, a red pixel followed by a green pixel in the firstrow 100 and, from left to right, a green pixel followed by a blue pixel,in the second row 101. This arrangement, in turn, results in, from topto bottom, a red pixel followed by a green pixel in the first column200, and, from top to bottom, a green pixel followed by a blue pixel inthe second column 201. This 2×2 sub-array 25 is repeated many times tocreate a large imager pixel array such as is shown in FIG. 3.

To correctly recreate the color image instant upon the imager array at afirst pixel, it is necessary to combine, in proper proportions, thesignal at the first pixel with a second and a third pixel. Specifically,the color signals from second and third pixels adjacent to an arbitrarychosen first pixel supply the additional color signals necessary tocorrectly demultiplex the first pixel color signal. If, for example, thefirst pixel is a green filtered pixel, then the color signals from theadjacent blue and red filtered pixels would be proportionally combinedwith the green pixel signal to correctly recreate the color for thegreen pixel location. It can be readily seen from the arrayconfigurations of FIGS. 2 and 3 that the color pixel signals from twolines or rows of the array are needed to fully recreate or demultiplexany given pixel.

Most CMOS imagers utilize either a random y or a random x and yaddressing architecture. These architectures allow the imagers to beread using either progressive or interlaced scanning methods. Theseimagers allow a single line of pixels to be read on any single access. Arow or line is selected and then read. Successive line reads are thenperformed to build a frame. In order to achieve full color reproductionfrom an imager, the current imager line data must be combined with apreviously read line as discussed above. Therefore, a memory must beused to store the results of previous imager line reads so that they canbe combined with current data.

Several prior art inventions relate to methods to read or to manipulateimage data. U.S. Pat. No. 5,742,325 to Curry et al describes a printerapparatus for rendering image data on a recording medium. Two channelsare used to read data such that data can be further processed accordingto one of two methods: half-toning for picture data and thresholding forline art or text. U.S. Pat. No. 6,243,100 B1 to Curry et al teaches animager processing system for interpolation performance.

SUMMARY OF THE INVENTION

A principal object of the present invention is to provide an effectiveand very manufacturable method to read a dual row read-capable CMOSimager.

A further object is to provide a method to perform color readout ofinterlaced scans on a dual read CMOS imager without requiring off-chipstorage of line, field, or frame data.

A further object of the present invention is to provide a method todouble the frame rate of a dual read per column CMOS imager.

A yet further object of the present invention is to provide additionalimage process functionality using a dual line read CMOS imager.

In accordance with the objects of this invention, a method of reading animager is achieved. The method comprises providing an imager arraycomprising n rows and m columns where a pair of rows can be read duringa single row access time. A first image field is completed bysequentially reading and storing pixel values of pairs of adjacent rowsof the imager array. The reading begins at a first row, and the readingstops when less than three rows are unread. Thereafter pixel values ofthe next row are read and not stored. Thereafter pixel values of thefirst row of the imager array are read and not stored. A second imagefield is completed by sequentially reading and storing pixel values ofpairs of adjacent rows. The reading begins at the second row, thereading stops when less than two rows are unread.

Also in accordance with the objects of this invention, a method ofreading an image field of an imager array is achieved. The methodcomprises providing an imager array comprising n rows and m columns. Apair of rows can be read during a single row access time. An image fieldis completed by a method comprising reading and storing a string ofpixel values for a pair of rows. The pixel values are copied for a firstrow of the pair from the pixel string to a first row register in anorder corresponding to the columns. The pixel values are copied for asecond row of the pair from the pixel string to a second row register inthe order corresponding to the columns. The steps of reading andstoring, copying the pixel values for a first row of the pair, andcopying the pixel values for a second row of the pair are repeated forall the rows of the imager array.

Also in accordance with the objects of this invention, a method ofreading an imager is achieved. The method comprises providing an imagerarray comprising n rows and m columns where a pair of the rows can beread during a single row access time. A first image field is completedby sequentially reading and storing pixel values of pairs of evennumbered rows of the imager array. The reading begins at a first evennumbered row, and the reading stops when all the even numbered rows havebeen read. A second image field is completed by sequentially reading andstoring pixel values of pairs of adjacent rows where the reading beginsat row 1 and where the reading stops when row n has been read.

BRIEF DESCRIPTION OF THE DRAWINGS

In the accompanying drawings forming a material part of thisdescription, there is shown:

FIG. 1 illustrates a CMOS imager array with dual row read per row accesscapability.

FIG. 2 illustrates basic Bayer pattern, 2×2 imager sub-array.

FIG. 3 illustrates a Bayer pattern, 8-row×12-column imager array forcolor imaging.

FIG. 4 illustrates a first preferred embodiment of the present inventionshowing a method to read a dual row read imager where the imagercomprises an odd number of rows.

FIG. 5 illustrates the first preferred embodiment of the presentinvention showing a method to read a dual row read imager where theimager comprises an even number of rows.

FIG. 6. illustrates a second preferred embodiment of the presentinvention showing how pixel values for each row of the dual row accesspair are preferably remapped.

FIGS. 7 through 9 illustrates the second preferred embodiment of thepresent invention for reading an imager array having dual row readcapability and showing alternative methods for scanning the imager.

FIG. 10 illustrates a fourth preferred embodiment of the presentinvention for reading an imager array where a part of the image fieldhas an increased dynamic range.

DESCRIPTION OF THE PREFERRED EMBODIMENTS

The preferred embodiments of the present invention disclose severalmethods to read a CMOS imager capable of dual row reading. It should beclear to those experienced in the art that the present invention can beapplied and extended without deviating from the scope of the presentinvention.

Referring now to FIG. 1, an imager array 10 is illustrated. Ofparticular importance, this imager array 10 has a dual row readingcapability. In a typical imager, an array of pixels is arranged as n-rowand m-columns. Each pixel senses incident light and converts the lightenergy into a voltage signal. The imager uses an analog-to-digitalconverter (ADC) to convert pixel voltages into digital data. Typically,a row of pixels can be read on any given access. During a read, theanalog-to-digital conversion is performed for each pixel in the row. Thedigital values of each pixel in the row is then read out through a databus as a string of data.

In the present invention, several unique methods are disclosed to read adual row read-capable imager. In a dual row read-capable imager, eachrow access to the imager performs an analog-to-digital conversion and adata streaming for two rows of the imager rather than a single row. Forexample, the APS75 CMOS imager from the Sarnoff Corporation contains twooutputs designed to provide for the readout of a pixel in two portions.Two identical correlated, double sampling circuits (CDS) are providedfor each column and two separate analog-to-digital converters areprovided for conversion of two rows simultaneously. The two samples aretime division multiplexed (TDM) to provide an output at twice thehorizontal clock output. This architecture can be exploited to performdual row readouts, rather than pixel signal partitioning, to therebyallow for several unique imaging methods that are disclosed in thepresent invention. The methods are applied to both color and gray-scaleimagers.

Referring again to FIG. 1, the dual row readout capable, imager array 10is shown in simplified form. Here, the array 10 is shown as an n-row bym-column array where n=8 and m=12. Each pixel 15 can be located as arow, column coordinate as shown. Of particular importance to the presentinvention, the imager array 10 can be accessed using a single dual-rowread to produce a single pixel value stream 20 as shown. In thisexample, row 2 and row 5 are selected for reading as a row pair. Thepixel values are presented to the imager data bus output at twice therate of the horizontal clock signal. In this way, it is possible to readout two rows of data in the time required for a single row access in atraditional imager.

A first preferred embodiment of the present invention comprises a uniquemethod to read and to process a dual-read capable imager for a colorimage. A basic Bayer sub-array 25 has been described above and isillustrated in FIG. 2. The Bayer pattern comprises a 2×2 pixel array 25of, from left to right, a red pixel followed by a green pixel in thefirst row 100 and, from left to right, a green pixel followed by a bluepixel, in the second row 101. This arrangement, in turn, results in,from top to bottom, a red pixel followed by a green pixel in the firstcolumn 200, and, from top to bottom, a green pixel followed by a bluepixel in the second column 201. This 2×2 sub-array is repeated manytimes to create a large imager pixel array such as is shown in FIG. 3.Referring again to FIG. 3, an imager array of pixels arranged in theBayer pattern is shown. The array shown in this example is an8-row×12-column array having eight rows: R1-R8 and 12-columns: C1-C12.

Referring now to FIGS. 4 and 5, the first preferred embodiment isillustrated in schematic form. In this method, an imager array 40comprises n rows and m columns. In this example, n=7 and m=6. Inpractice, much larger arrays comprising many thousands of pixels areused. Of importance, this imager array 40 is of the type where a pair ofrows can be read during a single row access time. As a first major step,a first image field is completed by sequentially reading and storingpixel values 42, 44, and 46 of pairs of adjacent rows (R1 and R2, R3 andR4, R5 and R6) of the imager array 40. The reading begins at a first row(R1), and the reading stops when less than three rows are unread. Inother words, if there are only one or two rows left to read, then theprocess of reading and storing data stops. In this case, the last row(R7) remains after the read and store of rows R5 and R6. Row R7 is thenext row, and it is read 48. However, the pixel values of the row R7 roware not stored 48. At this point, a first image field of data has beenread and stored.

Prior to reading the second image field, the first row R1 of the arrayis read and is discarded 50. A second image field is completed bysequentially reading and storing pixel values 52 and 54 of pairs ofadjacent rows. The reading of the second image field begins at thesecond row R2. The reading stops when less than two rows are unread. Inthis case, row pairs R2 and R3, and R4 and R5 are read and stored 52 and54. The last row pair R6 and R7 are read but discarded.

This method of reading the imager array results in two image fields: afirst image field and a second image field. Since two adjacent rows areread simultaneously during a single read time, adjacent rows, Rn andRn+1, are read out simultaneously using the dual readout structure. Thered, green, and blue signals are then available in columns Cm and Cm+1,permitting full color reproduction of the image detected by the imager.The method of this invention requires a dual readout structure toprovide two lines in parallel at the imager output. However, it does notrequire line or frame memories.

The imager array 40 does require a color filter array as shown. Thecolor filter array overlies the imager array. One of the color filtertypes overlies each pixel in the imager array 40. In this case, a Bayerpattern is used with three color filter types: red, green, and blue. Toconvert the first and second image fields into a color image, the pixelvalue of a first pixel having a first filter type is proportionallycombined with the pixel values of a second pixel having a second type offilter and of a third pixel having a third type of filter. Theconfiguration of the Bayer pattern allows adjacent pixels to be of eachof the three types. By reading the first and second image fields usingthe dual row accesses as shown in FIG. 4, data for the entire array isavailable for color conversion at twice the horizontal clock rate. Itshould be further noted that an equal number of row accesses is used foreach image field. This allows for an equal integration time for eachimage field. This is why some rows are read and then discarded such asrow R7 in the first field and rows 1, and 6 and 7 in the second field.

In the example of FIG. 4, the imager has an odd number of rows (n=7). Ifthe imager has an even number of rows, the method would appear as shownin FIG. 5. In this case, the array 40 has an even number of rows (n=6).The first field requires three reads 60, 62, and 64, and the secondfield requires three reads 66, 68, and 70. Again, the integration timesfor the first and second image fields are kept the same using “read &discard” steps 64 and 66. Using the first embodiment method, thesequential fields (first and second) are made up of interlaced scansthat are offset by one row and that are readout one field time from eachother.

Referring now to FIGS. 6 through 9, a second preferred embodiment of thepresent invention is illustrated. Referring particularly to FIG. 6, apixel value bit stream 401 is generated from the dual row access of theimager. The bit stream 401 comprises a data value (byte or word) foreach bit of each of the two rows. In general, the first row can belabeled Rn, and the second row can be labeled Rn+1. In the SarnoffCorporation APS75 CMOS imager, the pixel bit stream presents theconverted pixel values in alternating fashion. In particular, the pixelvalues are presented in ascending order from column 1 to column n andare presented alternating between the first row Rn and the second rowRn+1 of the pair. Therefore, the first data value is Rn/C1, the seconddata value is Rn+1/C1, the third value is Rn/C2, and so on. It is usefulfor the second preferred embodiment to re-map the data values such thatthe first half of the re-mapped bit stream 402 comprises the Row n data,from column 1 to column n, and the second half of the re-mapped bitstream comprises the Row n+1 data from column 1 to column n.

Another useful feature of this method are display registers 403 a and403 b for the re-mapped first and second rows, respectively. The twore-mapped lines are transferred from the re-mapped registers 402 a and402 b to the two-line storage element 403 a and 403 b for display. Whilethe next two lines are being reconstructed, the previous two lines areread out from the previous line. Therefore, the display rate is twotimes the imager line pair addressing rate. The imager can operate atnormal horizontal readout rates by employing a dual horizontal readoutto effectively double the imager pixel readout rate.

An image field is completed by the second embodiment method by readingand storing a string 401 of pixel values for a pair of rows. Then, thepixel values are copied for a first row of the pair from the pixelstring to a first row register 402 a in an order corresponding to thecolumns. The pixel values are copied for a second row of the pair fromthe pixel string to a second row register 402 b in the ordercorresponding to the columns. The steps of reading and storing, copyingthe pixel values for a first row of the pair, and copying the pixelvalues for a second row of the pair are repeated for all the rows of theimager array. These steps are repeated according to any of severalpossible patterns as shown in FIGS. 7, 8, and 9.

In particular, FIG. 7 shows how this method of reading and remapping thepixel data stream can be used to simply index down the imager 200. Thefirst two rows are read and remapped to the registers 204 in a firstaccess 202. Access continues 206 and 208 to fill the registers 208 and212 with the remaining rows until row n is read. As each new re-mappingis performed to fill the row registers 204, 208, and 212, the data canbe further shifted to the display registers, not shown, so that acontinuous display of image data can be achieved at twice the horizontalclock rate. In this progressive scan readout, the entire image frame canbe readout in line pairs in half the time of a single row access imager.In addition, each paired readout has an identical integration time.Therefore, if motion occurs, the displacement of an object in the scanwill occur every other line instead of on each line.

Referring now to FIG. 8, the second embodiment is used to scan theimager 200 from the upper and lower edges and toward the center. In thiscase, the first read and re-mapping 222 is of rows 1 and n to storefirst values 224. Subsequent reads 226 generate store values 228 andmove toward the middle of the array such that the last read and re-map230 is of row n/2 and row (n/2)+1 to generate the last stored values232. Referring now to FIG. 9, the second embodiment is used to scan theimager 200 as an upper half section and a lower half section. In thiscase, the first read and re-mapping 242 is of rows 1 and (n/2)+1 tostore first values 244. Subsequent reads 246 generate store values 248and move downward on the array such that the last read and re-map 250 isof row n/2 and row n to generate the last stored values 252.

Referring now to FIG. 10, a third preferred embodiment of the presentinvention is illustrated. Any arbitrary pattern of paired dual linereadouts can be used with the imager 300 designed with this dualline/column readout architecture. In this case, the line pairs do notneed to have the same optical integration time. In particular, in thethird embodiment a first image field is completed by sequentiallyreading and storing 304, 308, and 312 pixel values of pairs of evennumbered rows of the imager array 300. The reading begins at a firsteven numbered row (row 2), and the reading stops when all the evennumbered rows have been read (at row n if even array or n−1 if oddarray). A second image field is completed by sequentially reading andstoring 316, 320, and 324 pixel values of pairs of adjacent rows wherethe reading begins at row 1 and where the reading stops when row n hasbeen read (at row n−1 for an even array).

In the third embodiment, as shown, the first image field only containsthe even rows while the second image field contains both the even andodd rows. It is possible, using this method, to extend the dynamic rangeof the imager. Off-chip processing can generate an image with extendeddynamic range where the data is updated every frame. Only the even rowsare updated every frame while the odd rows (which can represent thedarker portions of the scene) are updated every other frame.

The advantages of the present invention may now be summarized. Aneffective and very manufacturable method to read a dual row read-capableCMOS imager is achieved. A method to perform color readout of interlacedscans on a dual read CMOS imager without requiring off-chip storage ofline, field, or frame data is achieved. A method to double the framerate of a dual read per column CMOS imager is achieved. Additional imageprocess functionality using a dual line read CMOS imager is provided.

As shown in the preferred embodiments, the novel methods of the presentinvention provide an effective and manufacturable alternative to theprior art.

While the invention has been particularly shown and described withreference to the preferred embodiments thereof, it will be understood bythose skilled in the art that various changes in form and details may bemade without departing from the spirit and scope of the invention.

1. A method of reading an imager, comprising: providing an imager arraycomprising n rows and m columns wherein a pair of said rows can be readduring a single row access time and output a single stream of pixelvalues from two rows of said imager which can be stored using a singlestorage register; completing a first image field by sequentially readingand storing pixel values of pairs of adjacent said rows of said imagerarray wherein said reading begins at a first said row and wherein saidreading stops when less than three said rows are unread; thereafterreading and not storing pixel values of next said row; thereafterreading and not storing pixel values of said first row of said imagerarray; and completing a second image field by sequentially reading andstoring pixel values of pairs of adjacent said rows wherein said readingbegins at second said row and wherein said reading stops when less thantwo said rows are unread.
 2. The method according to claim 1 whereinsaid step of reading and not storing pixel values of next said rowcomprises reading a pair of adjacent said rows starting with said nextrow.
 3. The method according to claim 1 wherein said imager arrayfurther comprises a color filter array.
 4. The method according to claim3 further comprising the step of completing a color image field bycombining said first image field and said second image field.
 5. Themethod according to claim 3 wherein said color filter array comprisesthree filter types.
 6. The method according to claim 5 wherein saidthree color filter types comprise red, green, and blue.
 7. The methodaccording to claim 5 wherein said three color filter types are arrangedsuch that each pixel of said imager array is filtered by one of saidcolor filter types and wherein any said pixel that is filtered by afirst said color filter type has complimentary said pixels comprising afirst adjacent said pixel that is filtered by a second said color filtertype and has a second adjacent said pixel that is filtered by a thirdsaid color filter type.
 8. The method according to claim 7 furthercomprising the step of completing a color image field comprising anarray of color pixel values wherein each said color pixel value isdetermined by combining said value of said pixel with said values ofsaid complimentary pixels.
 9. The method according to claim 1 whereinsaid first and second image fields comprise equal integration times. 10.A method of reading an imager, comprising: providing an imager arraycomprising n rows and m columns wherein a pair of said rows can be readduring a single row access time and output a single stream of pixelvalues from two rows of said imager which can be stored using a singlestorage register and wherein a color filter array overlies said imagerarray; completing a first image field by sequentially reading andstoring pixel values of pairs of adjacent said rows of said imager arraywherein said reading begins at a first said row and wherein said readingstops when less than three said rows are unread; thereafter reading andnot storing pixel values of next said row; thereafter reading and notstoring pixel values of said first row of said imager array; completinga second image field by sequentially reading and storing pixel values ofpairs of adjacent said rows wherein said reading begins at second saidrow and wherein said reading stops when less than two said rows areunread; and completing a color image field by combining said first imagefield and said second image field.
 11. The method according to claim 10wherein said step of reading and not storing pixel values of next saidrow comprises reading a pair of adjacent said rows starting with saidnext row.
 12. The method according to claim 10 wherein said color filterarray comprises three filter types.
 13. The method according to claim 12wherein said three color filter types comprise red, green, and blue. 14.The method according to claim 12 wherein said three color filter typesare arranged such that each pixel of said imager array is filtered byone of said color filter types and wherein any said pixel that isfiltered by a first said color filter type has complimentary said pixelscomprising a first adjacent said pixel that is filtered by a second saidcolor filter type and has a second adjacent said pixel that is filteredby a third said color filter type.
 15. The method according to claim 14wherein said step of completing a color image field comprises combiningsaid value of said pixel with said values of said complimentary pixels.16. The method according to claim 10 wherein said first and second imagefields comprise equal integration times.
 17. A method of reading animager, comprising: providing an imager array comprising n rows and mcolumns wherein a pair of said rows can be read during a single rowaccess time and output a single stream of pixel values from two rows ofsaid imager which can be stored using a single storage register;completing a first image field by sequentially reading and storing pixelvalues of pairs of even numbered said rows of said imager array whereinsaid reading begins at a first said even numbered row and wherein saidreading stops when all said even numbered rows have been read; andcompleting a second image field by sequentially reading and storingpixel values of pairs of adjacent said rows wherein said reading beginsat row 1 and wherein said reading stops when row n has been read. 18.The method according to claim 17 further comprising displaying saidfirst and second image fields such that said even numbered rows areupdated every frame and such that odd numbered said rows are updatedevery other frame.
 19. The method according to claim 17 wherein saidfirst and second image fields comprise integration times.